An automatic multi-view point cloud merging algorithm for structure light measure system was presented. Firstly, an improved region identification integrated with least-squares algorithm was used to compute the center of the reference point accurately, and the 3D coordinate of the reference point was ascertained through the theory of machine vision. Then, based on the invariability of spatial character, a rapid reference point matching algorithm was put forward to acquire the corresponding relationship between the reference points with different visual angles. Lastly, the rotation matrix and translation matrix were computed by quaternion decompose algorithm to carry out the automatic multi-view point cloud merging algorithm. The result of the experiments shows the validity and great practicability of the algorithm.